Measure-Valued Differentiation for Stationary Markov Chains
نویسندگان
چکیده
منابع مشابه
Measure-Valued Differentiation for Stationary Markov Chains
We study general state-space Markov chains that depend on a parameter, say, . Sufficient conditions are established for the stationary performance of such a Markov chain to be differentiable with respect to . Specifically, we study the case of unbounded performance functions and thereby extend the result on weak differentiability of stationary distributions of Markov chains to unbounded mapping...
متن کاملMeasure-Valued Differentiation for Markov Chains
This paper addresses the problem of sensitivity analysis for finite-horizon performance measures of general Markov chains. We derive closed-form expressions and associated unbiased gradient estimators for the derivatives of finite products of Markov kernels by measure-valued differentiation (MVD). In the MVD setting, the derivatives of Markov kernels, called D-derivatives, are defined with resp...
متن کاملPointwise stabilization of discrete-time matrix-valued stationary Markov chains
Let (Ω,F ,P) be a probability space and S = {S1, . . . , SK} a discrete-topological space that consists of K real d-by-d matrices, where K and d both ≥ 2. In this paper, we study the pointwise stabilizability of a discrete-time, time-homogeneous, stationary (p, P)-Markovian jump linear system Ξ = (ξn) n=1 where ξn : Ω → S. Precisely, Ξ is called “pointwise convergent”, if to any initial state x...
متن کاملStationary Probabilities of Markov Chains
In this paper, based on probabilistic arguments, we obtain an explicit solution of the stationary distribution for a discrete time Markov chain with an upper Hessenberg time stationary transition probability matrix. Our solution then leads to a numerically stable and eecient algorithm for computing stationary probabilities. Two other expressions for the stationary distribution are also derived,...
متن کاملKernel Estimation for Real-valued Markov Chains
Let {Xn : n = 0, 1, 2, ...} be a real-valued Markov chain. The purpose of this paper is to study properties of kernel type estimators of the stationary density and of the transition density of such a chain. These estimators were considered first by Roussas (1969) and by Rosenblatt (1970). Both authors extended to the Markov chain case some results for kernel estimators of the density based on a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Mathematics of Operations Research
سال: 2006
ISSN: 0364-765X,1526-5471
DOI: 10.1287/moor.1050.0171